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ABSTRACT 



An exploratory study of the situational and personal factors 
affecting the usage of a computer-based information service by 
health care professionals is described. The user population was 
free to use, or not to use, the service, and to use it as fre-- 
quently as they chose* It iB concluded that the relevance of the 
information to the immediate job-related problems of the user 
population is the major determinant of the first use of the 
service. Total Information needs, dissatisfaction with other 
sources of information, and beliefs about the satisfying nature 
of computer interaction and the reliability of computing machi- 
nery are predictive of more frequent use. 
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AN EXPLORATORT STUDY OF FACTORS AFFECTING USAGE OF AN ON-LINE 
COHPUTER-BASED BIBLIOGRAPHIC RETRIEVAL SERVICE 

^ INTRODUCTION 
<^C^his paper describes an exploratory study. The objective of 
tfe study was to explore factors which aight predict the usage 
ared non-usage of a computer-based bibliographic infornation re- 
trieyal systen, and for those who chose to use it, the anount of 
use they made of the system. It involves a population of users 
who are not nornally oriented toward using computer systems. 

The potential users of the service were not forced to use it 
to carry out their jobs. This situation is somewhat different 
from other studies of user acceptance of data retrieval systems. 
The information provided by the system described has varying 
value for different members of the potential user population. 
Some persons might find the ability to scan over a million jour- 
nal entries quite central to their jobs. Other persons might 
need this service only rarely, if at all. 

The service provided by th3 information retrieval system can 
also be obtained without the use of computers, at a good univer- 
sity library, for example. However, it requires much more time 
and effort to retrieve literature citations and abstracts manu- 
ally, but this must be balanced against the time and effort 
required to learn to use the computer-based information service. 

There are enough unique aspects to the setting and user 
population In this study to make confident predictions about user 
behavior impossible. For that reason, an exploratory approach 
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was taken. 

To study the possible determinants of use of this system, a 
set of general factors which logically Biight affect usage pat- 
terns were developed* Specific questions about these factors 
were administered to the potential user population via a pre-use 

and a later post-use questionnaire. 

I 

The data analysis of these rather extensive measurements 
proceeded more from general speculations about what ought to 
affect usage of the service than from formal hypotheses. The 
data analyses presented in the latter part of this paper are thus 
more descriptive than inferential, but they may represent some- 
thing of value to persons interested in providing information 
services for a population of users similar to those described 
below. 

Some G eneral F actors Which May Affect Use of Informs t ion 

Retr ieval Systems 

Physical Factors , The first requirement for use of a commu- 
nication system based on computer hardware Is physical access to 
the hardware* This very basic requirement is often assumed by 
those who studv man--machine interaction. In a realistic situa- 
tion, a user may have the choice of using some alternative to the 
computer information system. If the computer terminal is at a 
remote location, or if it is heavily used and the potential user 
must wait for access, the individual will probably be less likely 
to make use of the information system. The amount of use made of 



the computer system is probably dependent on a simple time and 
infornation cost/benefit function. In the case of bibliographic 
citations, if a library with printed indices is closer, and if 
the user feels that enough information can be obtained frooi this 
source in a reasonably short time period, the person will probab- 
ly not use the computer-based system. 

Eason and Damodaran (1981) discuss this time minimization in 
the context of determining interaction system requirements for 
task-oriented users, but its effect extends to initial access, as 
well. Sackman (1970) also relates the time to learn to access 
Information to the liklihood that use of the information will be 
made . 

Benefit Factors . Another quite basic requirement of a com- 
puter-based communication system is that the potential user must 
have some need for the service provided. These systems are often 
developed with little knowledge of the real information needs of 
the user population, and then presented to the users after deve- 
lopment is completed. If the service is subsequently not used, 
its failure may ♦hen be attributed to lack of "user-friendliness" 
or some other such structural factor, when in reality the service 
simply provides nothing that a potential user can use or could 
not obtain through other methods. Eason (1976) provides a very 
simple statement of this fact; 

"The task the person is performing defines his interests in 
information at that time." 

A new information system, and especially one which requires 
training to use, must solve some problem, or make some task 



easier for potential users, for it to be accepted and used. 

Satisfaction with Information Systems . This category is 
really made up of satisfaction with the information systems cur- 
rently being used, and satisfaction with the information system 
being introduced. 

The individual user's satisfaction with his current informa- 
tion collecting systems should predict usage. If the person is 
happy with the current system, there is no reason to expect that 
he/she will expend the energy and time to learn a new system. If 
he/she is unhappy with the status quo, we can expect more usage 
of the new system. 

The other type of satisfaction has more to do with the 
user's personality. Malde, quoted in Eason and Damordaran 
(1981), made a distinction between "thing-oriented" and "people- 
oriented" individuals. "Thing-oriented" persons were more effi- 
ciency centered, and felt that working with computer systems was 
more personally satisfying that did "people-oriented" indivi- 
duals. There is apparently some intrinsic satisfaction gained in 
using computers, beyond that of obtaining information from them, 
possibly because computers are seen as efficiency-enhancers. 

Satisfaction of users with experience may also change. 
Gaines and Shaw (1983) state experienced users often report 
frustration with a seemingly rigid computer system. Hiltz and 
Turoff (1978) describe a similar phase in experienced users, 
which they called the "saturation phase". They describe it as 
being produced by the evolution of new information needs within 



the user (partly because of experience with the current informa- 
tion system) which the system cannot meet. Satisfaction may thus 
be related to the general category discussed next, the user's 
prior experience with information retrieval systems. 

Prior Experience . Coombs and Alty (1981) point out that 
most computer-based data systems function primarily as one-way 
information sources, and do not adapt to the needs of individual 
users. The effectiveness of this kind of system is dependent 
upon the level of the user's knowledge. Persons with prior 
experience with computers, and especially with similar informa- 
tion systems, might reasonably be expected to make more use of a 
computer-based system than would a novice computer user. 

Beliefs about Computers . This area has received some atten- 
tion. Sackman (1970) found that non-users felt that computer 
hardware was "unreliable and untrustworthy; ... software is unna- 
turally difficult and requires too much time and painstaking 
attention". Hiltz and Turoff (1978) document an apparently com- 
mon belief that the computer will break if they do something 
wrong. During some pilot interviewing for this study, some 
individuals expressed beliefs that computers stifled creativity. 
Others showed a general distrust of machinery^ Including compu- 
ters. This was often expressed as a "machines are out to get »c** 
sentiment (the proposed name for this is the "Woody Allen Syn- 
drome" ) . 
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THE EXPLORATORY STUDY 



These general factors nay affect any user population's 
acceptance of a computer-based information service. This study 
was carried out to deteroine which of them operate in the 
specific setting of bibliographic information retrieval service 
available to a professional population. the aembers of which are 
naive or casual users of conputer services. 

The On-Line Service . 

MEDLINE is a computer-based data retrieval system which 
contains information about professional literature of interest to 
health professionals. Typical information is a complete litera- 
Lure citation with author, journal, etc., and a short abstract of 
the article. The data base is maintained by the National Library 
of Medicine, under the administrative umbrella of the National 
Institutes of Health. Over 3000 journals are indexed under 
25,000+ key terms. The data base contains over 1,000,000 en- 
tries, and is growing rapidly. 

MEDLINE was designed to permit searches by persons familiar 
with medical terminology, but not expert in computer operations. 
However, each user must be able to operate a computer terminal, 
dial into a timesharing service, enter account numbers and pass- 
words, etc. MEDLINE dialog is not a natural language system, so 
the user must also enter search requests in a fixed gramnjar, and 
must interpret sometimes cryptic responses from the system, MED- 
LINE is f^iirly typical of large data base information retrieval 



systems in this respect. While the MEDLINE dialog is much easier 
to use than a forsal data base language, it still would not be 
correct to call it "user-fri ndly". 

To help new users, there is a rather extensive tutorial 
program, MEDLEARN, which gives new users some experience opera* 
ting typical computer terminals (how to backspace, cancel com- 
mands, interrupt the MEDLINE mainframe computer's searches, 
etc.). Two large users* manuals are also provided. These give 
information about operational procedures, and also list the index 
terms . 

Many medical and research libraries also provide a MEDLINE- 
trained person, called the Technical Information Specialist 
(TIS), to help with searches. The TIS may sit with the user 
while the search is carried out, giving advice. Or the user 
night submit a search topic to the TIS, and have the TIS carry 
out all the computer operations. 

A user thus has the choice of three ways of using MEDLINE: 
I) an unassisted search; 2) an assisted search, where a TIS is 
present for advice; and 3) a submitted search where the TIS does 
all the computer interaction. 

The Study Objectives 

In 1980, NIH instituted an evaluation study of MEDLINE, 
focussing on its usefulness to professionals in communication 
disorders. Of particular interest were the determinants of usage 




of the service. Factors such as job categorjr, information needs, 
experience with computersp etc. were to be examined to deteriaine 
why some professionals used the MEDLINE service heavily, while 
others did not. Other evaluations of MEDLINE were also to be 
sade, but those will not be discussed here. 

The evaluation project began in September, 1980 and con- 
tinued unf.l March, 198S. The initial six months were devoted to 
developing a questionnaire to gather information about potential 
users, and to adainistering this questionnaire and training the 
potential users in MEDLINE procedures. The next 18 months were 
spent monitoring MEDLINE usage, and the final six months were 
used to process the data gained during the monitored usage phase. 

The evaluation project was carried out at five locations in 
the United States. A MEDLINE center was established at each 
location, and a full-time TIS was provided for each center. 
However, the data reported here were collected at the University 
of Connecticut only. The full final report, with data from all 
five centers, will be published shortly by the National Insti- 
tutes of Health. 

Stud ? Procedures 

The Sample, The potential user population at the University 
of Connecticut MEDLINE center consisted of 166 professionals in 
Language Pathology (36% of the total group). Speech Pathology 
(24%), Audiology (8%), with the remaining 32% spread among spe- 
cialties such as Aural Rehabilitation, Psychoacoustics , Otolaryn- 
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gology, etc. The group included practicing professionals as well 
as students pursuing advanced degrees (a little less than 22% of 
the total potential user group were students). This population 
was recruited prinarily during the initial six nonth start-up 
period, and during the first 12 nonths of the monitored usage 
period . 

Pre-Use Questionnaire . Before any searching was done, the 
user was asked to fill out an extensive pre-use questionnaire. 
This 13-page se] f -adsinistered questionnaire contained questions 
in the following areis: 

1. Job Profile. Questions in this section included primary 
and secondary joh interests, highest degree granted, percentage 
of tine spent in different job settings, such as clinics, hospi- 
tals, universities, etc. 

2. Information Needs. Potential users were asked hov fre- 
quently they needed updates froa various sources of professional 
information such as journals, books, continuing education, etc. 

3. Current Use of Infornation Sources. Respondents were 
asked how frequently they used each source of information to keep 
current and for cosprehensive reviews of their area of spec- 
iality. 

4. Satjisf action with Inforoation Sources. The respondent's 
satisfaction with each source of infornation was recorded on an 
8-point Likert-type scale. 




5. Experience with Autometed Inforoation System;*. Each 
potential user was asked about prior experience with other bib- 
liographic data base systeos such as ERIC (Educational Resources 
Inforaation Center), BRS (Bibliographic Retrieval Services), 
Lockheed Info Systems, etc. Both frequency of prior use and 
satisfaction with the system were recorded. 

6. BelieTs about Computers and Automated Information Sys- 
tems. Fourteen items, listed in Appendix A, were developed from 
the literature and from pilot interviews with potential users 
during the initial six-month period of the study. Each item 
represents a different belief, with no theoretical rational 
linki.^ ■ items, so each question was initially treated as a single 
indicator item in the data analysis. 

7. Physical Difficulty in Using the Service. Potential 
users were asked to estimate the travel time in minutes to the 
MEDLINE center. 

Post-Search Questionnaire . Each potential user who actually 
carried out a search was asked to fill out a short questionnaire 
immediately after completing the search. This questionnaire 
recorded the search mode (Unassisted, Assisted, or Submitted), 
the number of references generated, the user's satisfaction with 
these references, and the user's satisfaction with the TIS's 
help, if appropriate. 



Post -Use (Questionnaire . Each potential user who actually 
carried out a search filled out a post-use questionnaire which 
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contained items In the above 6 categories Identical (or nearly 
so — some minor changes were made in the wording of soae items bj 
request of the project director) to those in the pre-use 
questionnaire. This permitted some before-after comparisons. 
Potential users who did not carry out a search were asked to fill 
out an abbreviated questionnaire, contai-^ing some of the pre-use 
questions which allowed them to serve as a pseudo-control group 
for some analyses. 

An additional question was asked of users in the post-use 
questionnaire : 

8. Type of Training Received. Three types of training were 
available: Workshops, given at periodic intervals, TIS indivi- 
dual training sessions, available by appointment, and training by 
other MEDLINE users. Each user was asked to indicate the primary 
type of training received. 

RESULTS 

Users vs. Non-Users . 

Of the 166 potential users, 126 actually carried out one or 
more on-line searches. Thirty-eight did no searching, and two 
did not complete the questionnaire sequence sufficiently to per- 
mit their inclusion in the date analysis. 

Chi-square statistics based on cross-tebulat ions (for non- 
quantitative indicators) or t-tests (for qnsntita^^ive indicators) 
were used to determine which indicators in the eight qncKtlon- 




naire areas were significantly related to the decision to 
become a user or non-user of MEDLINE*s services. 

The data are most striking for their lack of predictive 
power. Only one interest area. Speech Pathology, had a slightly 
higher percentage of non-users than the other areas. Those with 
degrees granted before 1966 or after 1976 were slightly aore 
likely to use the service, but these groups are also the ones in 
which academic faculty and students are most likely to fall. 

The kinds of job tasks carried out by the respondents, the 
type of MEDLINE training received, and the travel time to the 
MEDLINE center showed no ability to discriminate between users 
and non-users. The appeal (and lack of appeal) of MEDLINE showed 
no evidence of being produced by differences in job requirements, 
training, or the difficulty in reaching the service. Neither did 
any of the opinion items about computer systems predict use/non- 
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In fact, the only item to clearly discriminate users from 
non-users was the frequency with which the respondent needed 
updates from journals, with those with higher frequency needs 
being much more likely to be users (t«2.40, p<.02). 

The dominant predictor was thus none of the items relating 
to ease of use, familiarity with computer systems, beliefs about 
computer systems, or training, but it is a rather important one: 
whether the respondent had a need for the primary service pro- 
vided by the computer-based communication system. 

15 
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Aaount of Searching Done by Users 

The object of this analysis was to identify factors which 
discriminated users who carried out a number of searches from 
those who carriec out few or no searches. Linear regression was 
used to predict the number of searches from a set ot indices 
which were constructed to summarize the individual indicators 
into iaore abstract variables. The constructed indices were: 

Total Frequency of Update (TOTUPj * This index was construc- 
ted by summing the responses to the 8 categories of Information 
needs in the pre-use questionnaire. It represents the respon- 
dent's total need for current information from all sources. 

Total Information Usage (TOTINT) . This index was construc- 
ted by summing the frequency of usage responses in the 12 infor- 
mation source categories of the preuse questionnaire. These 
responses were weighted as follows, before summing: 

"Every couple of years" «* 1 
"Annually" = 2 

"Every 3-6 months" = 6 

"Monthly" « 24 

"More than once a month" » 48 

Total Use of Information Bases (TOTtSE) . This index was 
constructed by summing the number of times per year the respon- 
dent used automated information retrieval services as reported 
in the pre-use questionnaire. It thus represents prior experi- 
ence with information retrieval systems. 
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Total Satisfaction with Sources of Informa tion (TOTSAT) . 
This index was constructed by summing the level of satisfaction 
reported by respondents for all 13 possible sources listed in the 
pre~use questionnaire. 

lit addition to these constructed indices, a number of indi- 
vidual variables were included as potential predictors of the 
level of searching carried out. All were taken from the pre-use 
questionnaire. They were: 

The 14 computer and information system belief items (see 

Appendix A). 

The number of minutes travel time to the MEDLINE center. 

— The year the highest degree was granted. 

Type of training (from post-use questionnaire) 

a) workshop 

b) instruction from a friend 

c) by TIS 

Professional interest in Speech Pathology (primary, 

secondary, or not at all) 

Although some of these items did not discriminate between users 
and non-users, an examination of zero-order product-moment corre- 
lations indicated reasonably high covariance between the indivi- 
dual items selected and the number of searches carried out by an 
individual . 

Stepwise regression was used in an txploratory analysis to 
identify the strongest predictors of the amount of searching 
done. Table I shows the results cf this regression. 

Four statistically significant predictors were found. In 
concert, they explain 12% of the variance in the noraber of sear- 
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ches. This iigure Is stignificant (p < .001). Each of the par- 
tial statistics for the individual predictors are also signifi- 
cant at p < .05. None of the additional predictors reached 
significance. In addition, a sinultaneous regression which in- 
troduced all predictors in the same equation isolated the sane 
four predictor variables as the onlj significant ones. Taken 
together, these results indicate that the four variables are the 
optimal set for predicting the number of searches carried out. 

The variable vith the strongest predictive power vas the 
total information usage index (TOTINT). High information usage 
of other types of information sources was predictive of more 
frequent MEDLINE searches. Apparently there exists a group whose 
members are high consumers of information. This group is more 
inclined to use MEDLINE. 

The second most predictive variable was one of the belief 
items: 

"Playing with computers can be fun and personally 
satisfying .** 

Agreement with this item was predictive of a higher number of 
searches. It may reflect a positive orientation toward automated 
information systems* '>r perhaps it is reflective of the possible 
higher acquaintence of the high information consumers with compu- 
ter equipment. 

The third item was also a belief item: 

**CoiDputer8, like most machineSt always break down when you 
need them most." 
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Agreeisent with this item predicted sore searches. This may be an 
unfortunate side-effect of reliance on a tiffle-sharing computer 
system for critical information. Persons likely to do sore 
searching are also those most likely to have been disappointed by 
equipment and system failures. 

The last predictive item vas the total satisfaction with 
sources of information (TOTSAT) Index. The lower this index, the 
higher the number of searches the respondent was likely to carry 
out. Apparently those dissatisfied with more traditional sources 
of information will use newer services such as HEDLINE to seek 
satisfaction . 



CONCLUSIONS 

In this exploratory study, we reach the conclusion that the 
best predictor of initial use of a computer-based information 
service is a felt need for the information supplied. We find no 
evidence of the predictive power of prior experience, personality 
or belief variables, or even the extensiveness of training 
received • 

A composite picture of the more frequent user of MEDLINE 
emerges from the analysis of those who actually made the decision 
to use the service. Re/She is a high consumer of information, 
but one who is somewhat dissatisfied with current sources of 
information. He/She regards computer equipment positively, but 
is frustrated by unreliablity in the systems. 
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To the degree that the user population described In this 
paper is representative of other task-oriented professionals who 
are not primarily interested in conputers, but in the infornation 
which they can provide, we nay draw soae tentative conclusions: 

1) Any computer-based information service nust provide 
information directly related to the jobs carried out by the 
professionals. If they do not recognize the need for the infor- 
mation service, they will not even sample the service. If the 
potential user population is merely unaware of the value of the 
system, an educational program might be called for. But if the 
service cannot easily be shown to be valuable to the current 
tasks being carried out by the professionals, it is unlikely to 
be accepted. 

2) Those who really see the need for the service will use 
it despite inconveniences, and they will u»e it even if they have 
had little or no prior experience with computer systems. 

3) Increasing the reliability and availability of the in- 
formation service will increase use of the system. 

4) Increasing compater literacy and decreasing fear of 
computing machinery by educational programs should increase use 
of the information system. 
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TABLE I 



NUMBER OF USES OF MEDLINE PREDICTED BY USER CHARACTERISTICS 



Predictor 
Variable 

Total Inforsation Usage 
(TOTINT) 

Total Satisfaction with 
Sources of Information 
(TOTS AT) 

"Playing with computers 
can be fun..." (Agree) 

"Computers . . • always 
break down Just when you 
need them" (Agree) 



Beta 
Weight 

.232 

- .183 



215 
,186 



F for Beta Sig. 



7.57 



4.67 



6.98 



5.37 



< .01 



< .05 



< .01 



< .025 



d.f. for Betas «: 1;149 



Multiple R - .343 

F for Regression = 4.87 (4;146 d.f.) p < .001 
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APPENDIX A 

BELIEFS ABOUT COMPUTERS AND AUTOMATED INFORMATION SYSTEMS 



1. Computers are expensive pieces of sophisticated equipment 
that should be utilized only by trained experts. 

2. Recent technology has made computer operation so simple even 
a layman can put them to good use. 

3. Computers may be good for other people, but I personally have 
no need to use one. 

4. In today's society, everyone should know how to operate a 
computer . 

5. If 1 use a computer, I am afraid I might damage 

6. Computers, like other machines, always break cjwn just when 
you need them motjt. 

7. It takes too long to learn to use a computer for it to be 
useful to me, 

8. Computers are detrimental because they reduce creative 
thinking and restrict human development. 

9. Playing with computers can be fun and personally satisfying. 

ID. Raving a computer-based information system available provides 
Communication Disorder Specialists with an additional, useful 
tool . 

11. Having a computer-based information sytem would provide me 
with an additional useful tool in my profession. 

12, Computers are beneficial because they save t1»»> »f>d give 
professionals greater freedom to investigate new ideas. 

13. Computer systems and languages are just too complicated to 
use for relatively simple jobs like reviewing professional 
literature. 

14, Computer information systems reduce the danger that I will be 
unaware of an important advance in my speciality. 

Answered on as 8-point scale, ranging from 1-DlSAGREE to 8«AGREE 
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